Method and system for monitoring communication between vehicle and remote terminal

ABSTRACT

This disclosure relates to a method (S0) for communicating information between a vehicle (20) and a remote terminal (10), the method comprising the steps of: providing (S1) at least one image frame (11) representing at least a portion of the vehicle&#39;s surrounding; wherein an encoded pattern (ENP1) representing predetermined information to be transmitted to the remote terminal (10) is visible on the at least one image frame (11); transmitting (S2) the at least one image frame from the vehicle (20) to the remote terminal (10); reading (S3), at the remote terminal (10), the encoded pattern (ENP2) visible on the at least one image frame (11); decoding (S4) the predetermined information from the encoded pattern (ENP2) read at the remote terminal. A system for carrying out the method is provided also.

TECHNICAL FIELD

The present disclosure relates to a method and system for monitoringcommunication between a vehicle and a remote terminal, and in particularmethods and systems for monitoring communication and for communicatinginformation in remote-operated vehicle systems.

BACKGROUND

In remote-operated vehicle systems, a vehicle is configured with sensorsfor capturing sensory data of the surroundings of the vehicle. Thesensory data is then transmitted to a remote terminal at which arepresentation of said surroundings is constructed based on said sensorydata. By means of the representation, the vehicle may be operated fromthe remote terminal, for instance by a human operator, to navigate itssurroundings.

In order to provide an industrially applicable method and system forcommunicating information in remote-operated vehicle systems, certainaspects with regards to safety and security must be adequately solved.One particular aspect relates to the risk that the vehicle is operatedbased on inadequate representations of the vehicle's surroundings.Inadequate representations may for instance occur at the remote terminalwhen the image-to-image latency is too high resulting in vehicleoperations too late in response to changes within the vehicle'ssurrounding. Inadequate representations may also occur if sensory datafrom the vehicle is not transmitted correctly, thus leading to erroneousrepresentations of the vehicle's surrounding, or if the representationis somehow erroneously rendered at the remote terminal, or if thesensory data as captured is inadequate itself, for instance due to amalfunctioning sensor.

Some present inventions attempt to solve some of these issues byimplementing communication systems with sufficient speed and redundancy.Such systems, however, may become very complex and expensive and may notaddress the risk of inadequate presentations of a vehicle's surroundingin a satisfactory manner, despite timely and correctly transmittedsensory data.

Thus, there is a need for an invention which at least improves uponpresent inventions in this regard, and in particular a system and methodfor reducing the risk that a representation displayed at a remoteterminal is an inadequate representation of the vehicle's surrounding.

SUMMARY

It is an object of the present invention to provide an improved solutionthat alleviates at least some of the mentioned drawbacks with presentsolutions. Furthermore, it is an object to provide a method and systemfor communicating information between a vehicle and a remote terminal,in particular to provide an improved method and system with the objectto reduce the risk that a representation displayed at the remoteterminal is an inadequate representation of the vehicle's surrounding.

By inadequate representation of the vehicle surrounding, it may be meantthat the constructed representation does not depict a sufficientlyadequate representation of the vehicle surrounding so as to ensure safevehicle operation.

The invention is defined by the appended independent claims, withembodiments being set forth in the appended dependent claims, in thefollowing description and in the drawings.

According to a first aspect of the invention, a method for communicatinginformation between a vehicle and a remote terminal is provided. Themethod comprises the steps of providing at least one image framerepresenting at least a portion of the vehicle and/or the vehicle'ssurrounding, wherein an encoded pattern representing predeterminedinformation to be transmitted to the remote terminal is visible on theat least one image frame; transmitting said at least one image framefrom the vehicle to the remote terminal; reading at the remote terminal,the encoded pattern visible on the at least one image frame, anddecoding the predetermined information from the encoded pattern read atthe remote terminal.

By transmitting predetermined information in this manner, i.e. as anencoded pattern visible on the at least one image frame, it may increasethe probability that the image frame containing the encoded pattern isan adequate representation of at least a portion of the vehicle'ssurrounding. In particular, it may increase the probability that the atleast one image frame has been correctly transmitted and rendered at theremote terminal. If decoded information does not correspond to thepredetermined information which was to be transmitted to the remoteterminal, or if the encoded pattern is partially or entirely missing, itmay be deduced that an error has occurred either during the transmissionof the at least one image frame or during the rendering of the at leastone image frame. This enables a safer and more secure method forcommunicating information between a vehicle and a remote terminal, sinceit may prevent that the vehicle is operated based on at least partlyinadequate representations of the vehicle surrounding as displayed atthe remote terminal.

Information communicated between the vehicle and the remote terminal maybe communicated via one or more communication links established betweenthe vehicle and the remote terminal. Image frames provided with theencoded pattern may be compressed and sent as one or more packages in abitstream to the remote terminal via the one or more communicationlinks. Then, at the remote terminal, the one or more packages areunpackaged, and the data thereof is used for constructing arepresentation of the vehicle's surrounding. The communication links maybe over separate communication networks, or over different links orchannels in a common communications network. The communication links maybe wireless communication links established over a cellular network,such as 4G/LTE or 5G.

By representation, it may be meant a video feed displayed on one screenat a remote terminal. It may also refer to a plurality of video feedportions displayed on separate screens at a remote terminal whichtogether form a video feed. Thus, each video frame of the video feed maybe based on a respective at least one image frame transmitted from thevehicle to the remote terminal. Each image frame may be associated to aspecific time instance and a specific imaging sensor of the vehicle. Bycombining sensory data from a plurality of imaging sensors, a pluralityof image frames for every time instance may be transmitted to the remoteterminal and used in constructing the representation. The plurality ofimaging sensors may be arranged so as to capture visual information inspecific directions, for instance in a front view direction, a rear-viewdirection, in side view directions etc. The representation may alsoincorporate information from additional sensory data, such as soundinformation, distance measurement and vehicle data, or augmentedfeatures based on such information.

The predetermined information may be a random generated string ofsymbols. The predetermined information may be a string of symbols whichchange over time in a predictable manner. The predetermined informationmay change over time so that it changes for every time instance an imageframe is captured. For instance, the vehicle imaging sensors may beconfigured to capture imaging frames 60 times per second of the vehiclesurrounding. The predetermined information encoded onto each image frameas the encoded pattern may then change between each consecutive imageframe. Moreover, the predetermined information may be unique for everytime instance at which an image frame is captured. The change ofpredetermined information encoded to encoded patterns may besynchronized with the image capturing by the one or more vehicle imagingsensors, e.g. by using a synchronization signal shared by the vehicleimaging sensor(s) and the unit generating the predetermined informationand/or the encoded pattern. By synchronizing the image capturing and thegeneration of the encoded pattern, it is ensured that no encoded patternis viewed at the vehicle but not captured in any image frame and therebynot transmitted to the remote terminal.

The predetermined information may be encoded into an encoded patternbased on a predetermined encoding rule shared between the vehicle andthe remote terminal. The encoded pattern may be any suitable patternwith sufficient complexity for encoding the predetermined information.The encoded pattern may be a string of symbols. The encoded pattern maybe a sequence of elements which can be changed between different states.The encoded pattern may be visible at a general single location of eachimage frame, e.g. in a predetermined quadrant or corner of the imageframe or along a predetermined image frame border of the image frame.The encoded pattern may comprise a plurality of portions visible atgenerally different locations of each image frame, e.g. in apredetermined quadrant, corner etc. of the image frame or along apredetermined image frame border of the image frame.

According to one embodiment, the predetermined information of theencoded pattern may include a timestamp of the at least one image frame.By having the predetermined information at least including a timestampof the at least one image frame, it may be possible to determine if aspecific image frame displayed is too old and adequate measurements maythereafter be taken. The age of an image frame, once determined afterthe predetermined information has been decoded, may be compared to anestablished theoretical or standard or average or mean image-to-imagelatency of the system. If the age of the image frame is determined to beoutside a predetermined threshold limit of the image-to-image latency ofthe system, adequate actions may be taken, for instance, reducing thespeed of the vehicle or stopping it entirely. The predeterminedthreshold limit may be set based on configuration of e.g. sensors,screens, communication units and communication network used. Thethreshold for an acceptable image-to-image latency may in one embodimentbe between 80-250 ms. In another embodiment, the threshold may bebetween 100-200 ms.

According to one embodiment, the method may further comprise the step oftransmitting sensory data including the at least one image frame fromthe vehicle to the remote terminal; and transmitting drivinginstructions from the remote terminal to the vehicle together with thedecoded predetermined information and/or the read encoded pattern. Thevehicle may be configured to only accept driving instructions withreferences to decoded predetermined information or to a read encodedpattern. By this, the vehicle may verify that driving instructions arein fact based on correct or at least adequate sensory data and may thusaccept them. The encoded pattern may be decoded at the remote terminalto determine the predetermined pattern. In that case, the predeterminedinformation decoded from the read encoded pattern may be transmittedtogether with the driving instructions. Alternatively, or additionally,the read encoded pattern may be transmitted together with the drivinginstructions. In that case, the encoded pattern may be decoded at thevehicle to determine whether the driving instructions are based oncorrect received information.

According to one embodiment, the method may further comprise the step ofdetermining an image-to-image latency or a round-trip time between thevehicle and the remote terminal based on the decoded predeterminedinformation of the at least one image frame. The predeterminedinformation encoded on each image frame may include a timestamp of saidimage frame. Thus, when decoding the encoded pattern of thepredetermined information, the timestamp of the image frame may bediscerned and compared to the present time by which an image-to-imagelatency may be determined. The round-trip time may be determined at thevehicle. When receiving driving instructions from the remote terminal inreference to some decoded predetermined information including atimestamp of a transmitted image frame, the timestamp may be compared tothe current time.

According to one embodiment, the method may further comprise the step ofexecuting an emergency fail-safe procedure of the vehicle when theimage-to-image latency or the round-trip time fails to satisfy apredetermined image-to-image latency condition or a predeterminedround-trip time condition. By this, a safe vehicle operation may beensured.

According to one embodiment, the method may further comprise the stepsof generating a message authentication code based on the encoded patternof at least one image frame and using the message authentication code toauthenticate communication transmitted between the vehicle and theremote terminal. The vehicle and the remote terminal may share a messageauthentication code algorithm and a message authentication code key. Themessage authentication code key and the predetermined informationencoded as an encoded pattern or the encoded pattern itself may be usedas an input to the message authentication code algorithm so that a firstmessage authentication code is generated. The message authenticationcode may be transmitted together with the image frame provided with theencoded pattern from the vehicle to the remote terminal. At the remoteterminal, the predetermined information as decoded or the encodedpattern may be used together with the shared message authentication codekey as input to the shared message authentication code algorithm bywhich a second message authentication code is generated. The first andsecond message authentication codes may then be compared to see if theyare identical or not. The same procedure may be performed in theopposite direction, from the remote terminal to the vehicle. Thus, itmay prevent relying on information communicated between the vehicle andremote terminal which have been tampered with, thus resulting inimproved security.

According to one embodiment, the encoded pattern comprises a set ofpattern elements configured so that each pattern element may be set toeither of at least a first state and a second state. The number ofpattern elements may be selected so as to achieve a sufficientcomplexity in predetermined information which can be encoded. Moreover,the pattern elements may be configured to be able to be set to aplurality of states, i.e. two or more than two, so as to increasecomplexity of the encoded pattern for any given number of patternelements in the encoded pattern. For example, the number of patternelements may be anything between 2 to 30 or more, and the number ofstates of each pattern element may be anything between 2 and 100 ormore. Thus, the encoded pattern may encode predetermined information ofgreat complexity.

According to one embodiment, the encoded pattern may be a color pattern,and the at least a first state and a second state of each patternelement may be of different colors. The colors may be predetermined andappropriately selected in view of the colors present in the image frame.The colors used in the encoded pattern may be selected so as to beeasily discernible relative the background in image frames. The colorsused may be “unnatural” colors, i.e. colors which do not commonlyappearing in nature.

According to one embodiment, the encoded pattern may be generated by alight emitting device placed in the field of view of an imaging deviceof the vehicle configured to capture sensory data used for generatingthe at least one image frame. By this, the encoded pattern will beprovided onto each image frame in an efficient manner. The encodedpattern need not be processed onto the images afterwards using softwaremeans. The light emitting device may be a display configured to providethe predetermined information as the encoded pattern. The display may bean LCD display, OLED display or any other suitable display comprising aset of pixels which can be adjusted in terms of brightness and color.The light emitting device may be configured to generate an encodedpattern of predetermined information every time instance where imageframes of the vehicle's surrounding is captured. The light emittingdevice may be arranged inside the vehicle or outside the vehicledepending on where a particular imaging sensor is arranged. For example,if an imaging sensor is arranged inside the vehicle and aimed to captureimage frames of a front view through the front window, the lightemitting device may be arranged on or near the instrument board of thevehicle such that it is visible in the field of view of the imagingsensor.

According to one embodiment, the light emitting device comprises aplurality of light emitting diodes configured to provide patternelements of the encoded pattern. The light emitting diodes may be of anycolor, for instance red, green, blue and/or white. The light emittingdiodes may be organic light emitting diodes. The light emitting diodesmay form an RGB set which together provide a pattern element. By this,the color of each pattern element may be adjusted.

According to one embodiment, the hue, brightness and/or saturation ofavailable colors of each pattern element may be adjusted based onambient light conditions of the vehicle. Different combinations of hue,brightness and/or saturation may form one or more states of each patternelement. The vehicle may be configured with an ambient light detectorfor monitoring the ambient light. The ambient light detector may bearranged to monitor the ambient light conditions in which the lightemitting device must operate in to provide the encoded pattern. Forexample, in case of strong sunlight, the light emitting device may beadjusted so as to provide an encoded pattern visible in the image frameswhich is more easily discernable given the ambient light conditions.Moreover, the number of states of each pattern element may be adjustedbased on ambient light conditions.

According to one embodiment, the encoded pattern may be read at theremote terminal by means of an encoded pattern detection device. Theencoded pattern detection device may be one or more light sensor and/orcolor sensor. The encoded pattern detection device may further comprisea plurality of one-pixel cameras, each arranged to detect one pixel ofthe image displayed to the user, each such pixel displaying one patternelement. The sensor, sensors or cameras may be arranged directly on ascreen on which the image frame is displayed to the user. By this,automatic reading of the encoded pattern may be enabled. The encodedpattern detection device may be configured to transmit detected encodedpatterns to a decoding unit configured decode the detected encodedpattern. The decoding unit may be at the remote terminal, or at thevehicle. If the decoding unit is arranged at the vehicle, the encodedpattern detected by the encoded pattern detection device may betransmitted to the vehicle, either alone or together with e.g. drivinginstructions to the vehicle.

According to one embodiment, the encoded pattern may be embedded intothe bitstream that transmits the at least one image frame using softwareinstead of by means of a light emitting device. By this, the principleof the encoded pattern may still be used for an image frame. This may beadvantageous in circumstances where a light emitting device may noteasily be arranged in the field of view of an imaging sensor. Accordingto one embodiment, the method comprises capturing image frames by meansof different imaging sensors, wherein a light emitting device isarranged in the field of view of one imaging sensor to provide a visibleencoding pattern while for image frames of a different imaging sensor,the encoded pattern is embedded into the bitstream that transmits thatimage frame using software.

According to one embodiment, the encoded pattern may be encrypted, andthe encoded pattern is decrypted using an encryption key. By this, thepredetermined information may not be easily determined by a third partyother than the vehicle and the remote terminal. Moreover, it may hinderthird parties from taking unauthorized control over the remote operatedvehicle or transmit to the remote terminal. The decryption of theencrypted encoded pattern may be performed at the remote terminal.Alternatively, or additionally, the decryption may be performed at thevehicle after the transmitted encrypted encoded pattern has been read atthe remote terminal and the read pattern has been transmitted back tothe vehicle. In such embodiment, only the vehicle needs to know theencryption key and still be able to verify e.g. the image-to-imagelatency or the round-trip time.

According to a second aspect of the invention, a communications systemis provided. The communications system may comprise a first system partfor use in a vehicle. The communications system may comprise a secondsystem part for use in a remote terminal. The first and second systempart may be configured to enable the method for communicatinginformation between the vehicle and the remote terminal according to thefirst aspect of the invention or any embodiments thereof.

The first system part may comprise one or more light emitting devicesconfigured to be arranged in the field of view of one or morecorresponding imaging sensors. The first system part may comprise saidone or more corresponding imaging sensors. The first system part maycomprise additional sensors for capturing additional sensory data of thevehicle's surrounding or the vehicle itself. The first system part maycomprise a communications unit configured to communicate with a remoteterminal. The first system part may comprise a processing device forpackaging, compressing, and preparing the information to be communicatedto the remote terminal. The processing device may likewise be configuredto process information communicated to the vehicle from the remoteterminal. The processing device may be configured to enable thedisclosed method or any of its embodiments.

The second system part may comprise communications unit configured tocommunicate with the vehicle. The second system part may comprise aprocessing device configured to process the information communicatedfrom the vehicle to the remote terminal. The second system part may beconfigured to transmit the information, including the at least one imageframe to a display at the remote terminal for displaying said at leastone image frame with the encoded pattern. The second system part maycomprise an encoded pattern detection unit for detecting the encodedpattern. The detected encoded pattern may be transmitted to decodingunit to decode the encoded pattern.

According to a third aspect of the invention, a vehicle comprising thefirst system part according to the second aspect of the invention isprovided. The vehicle may be an automated vehicle. The term “automatedvehicle” may refer to any automated vehicles according to SAE Level 1 toSAE Level 5 of the SAE International published classification systemwith six levels—ranging from fully manual (SAE Level 0) to fullyautomated systems (SAE Level 5), known as J3016_201609, or Taxonomy andDefinitions to On-Road Motor Vehicle Automated Driving Systems. Thus, byautomated vehicle, it may refer to any such classified vehicles whichare configured to be remote controlled from a remote terminal, forinstance by a human operator, either all the time or momentarily.

According to a fourth aspect of the invention, a remote terminalcomprising the second system part according to the second aspect of theinvention is provided.

According to a fifth aspect of the invention, a vehicle systemcomprising one or more vehicles, at least one remote terminal and acommunications system according to the second aspect of the invention.

Apart from image frames, the representation may be constructed based onadditional sensory data representing additional information about thevehicle surroundings, such as sounds, range information, temperature,humidity etc. Moreover, in conjunction with image frames beingtransmitted between the vehicle and remote terminal as disclosed in thefirst or second aspect, the method and system may also transmitadditional sensory data of the vehicle's surrounding and/or the vehicleitself. The sensory data may be visual information, sound distanceinformation via for instance radar or laser, temperature, speed data,motor data etc.

Moreover, the representation may be rendered by means of a remoteterminal VR headset connected to the remote terminal.

Moreover, the rendered representation may present a field of view in adriving direction of the vehicle having a horizontal field of view thatextends between 90 degrees and 360 degrees and having a vertical fieldof view that extends between 45 degrees and 180 degrees.

Moreover, the displayed rendered representation may present augmentedreality features for assisting a user operating the vehicle from theremote terminal. The augmented reality features may be based oninformation transmitted from the vehicle.

Moreover, the vehicle may be a land vehicle, such as a motor vehicle ora railed vehicle. The vehicle may further be a flying vehicle or afloating vehicle.

A plurality of embodiments and aspects of the invention have beenbriefly summarized. Although embodiments of the invention in certaincases are disclosed separate from each other, it should be understoodthat these embodiments may be claimed in various combinations, unless itis apparent that certain disclosed embodiments are mutually exclusive.

The invention is defined by the appended independent claims, withembodiments being set forth in the appended dependent claims, in thefollowing description and in the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will in the following be described in more detail withreference to the enclosed drawings, wherein:

FIG. 1 shows a schematic view of a system according to an embodiment ofthe present invention;

FIG. 2 shows a perspective view of a remote terminal according to anembodiment of the present invention;

FIG. 3 shows an image frame with an encoded pattern as provided in amethod according to an embodiment of the present invention;

FIG. 4 shows an illustrative view of an encoded pattern according to anembodiment of the present invention;

FIG. 5 shows a flow chart of a method according to an embodiment of theinvention;

FIG. 6 shows a flow chart of a method according to an embodiment of theinvention;

FIG. 7 shows a flow chart of a select collection of method stepsaccording to an embodiment of the invention,

FIG. 8 shows a flow chart of a select collection of method stepsaccording to an embodiment of the invention.

DESCRIPTION OF EMBODIMENTS

The present invention will be described more fully hereinafter withreference to the accompanying drawings, in which preferred embodimentsof the invention are shown. This invention may, however, be embodied inmany different forms and should not be construed as limited to theembodiments set forth herein; rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art. In thedrawings, like numbers refer to like elements.

FIG. 1 illustrates a system according to an embodiment of the presentinvention. A remote terminal 10 is communicatively connected to one ormore vehicles 20 via a communications network 40. As seen in FIG. 2 ,the remote terminal 10 may comprise a display unit 11 and a control unit13. On the display unit 11 image data, i.e. at least one image frame forevery time instance captured, is displayed. The image data can beprovided by one or more imaging sensors of the vehicle, which may bearranged to capture visual information in various directions relativethe vehicle. In order to provide a wide image of the vehicle'ssurrounding, image data from a plurality of sensors may be combined. Theimage data may also comprise other sensory data, or be communicatedalong with other sensory data, that can be displayed to the user at theremote terminal 10. The other sensory data can be presented on top ofthe image data on the display unit 11 to provide an augmented realityview for the user. A user uses the control unit to generate informationto be communicated to the vehicle 20, such as driving instructions. Thedriving instructions may include steering instructions, acceleration orbraking instructions and similar. When receiving image data from thevehicle using a communication method as in the present invention, eachimage frame is provided with an encoded pattern ENP visible on said eachimage frame. As can be seen in FIG. 2 the encoded pattern ENP isprovided by means of a light emitting device 21 arranged in the field ofview of the one or more imaging devices by which the displayed imagedata is captured. The light emitting device 21 comprises a set of lightemitting units 211, 212 which can be independently adjusted in terms ofbrightness and optionally color. The set of light emitting units 211,212 are configured so as to provide the encoded pattern ENP in terms ofpattern elements ENPa, ENPb. The encoded pattern ENP is read by anencoded pattern detection device and the detected encoded pattern isdecoded by a decoding unit.

As see in FIG. 1 , one remote terminal 10 can be in communication withseveral vehicles 20. The remote terminal 10 may then be used formonitoring the plurality of vehicles 20, and if necessary assume controlof a specified vehicle 20.

FIG. 3 an image frame 12 with an encoded pattern ENP as provided in amethod according to an embodiment of the present invention. The encodedpattern ENP may comprise of a plurality of pattern elements ENPa, ENPbwhich are arranged in some formation, for instance in series as shown inFIG. 3 . Depending on the circumstances of the visual information to berepresented as an image frame or of the imaging sensor providing saidimage frame, different positions within the image frame for the encodedpattern may be favorable. For instance, the encoded pattern may beprovided at a single general location within the image frame 12, e.g.general location 12 a towards the lower right corner. The encodedpattern may be provided in one of various general locations 12 a, 12 b,12 c, 12 d within the image frame 12. The encoded pattern may bedistributed over various general locations 12 a, 12 b, 12 c, 12 d withinthe image frame 12.

FIG. 4 shows an illustrative view of an encoded pattern ENP, ENP′according to an embodiment of the present invention at two differenttime instances, wherein the first encoded pattern ENP is associated witha first time instance and the second encoded pattern ENP′ is associatedwith a second time instance following the first time instance, forinstance the next time instance at which imaging sensors are configuredto capture the at least one image frame. The encoded pattern ENP, ENP′comprises a plurality of pattern elements ENPa-ENPh arranged in series.The encoded pattern ENP in FIG. 4 is shown to be comprised of eightpattern elements ENPa, ENPb, ENPc, ENPd, ENPe, ENPf, ENPg, ENPh but mayof course comprise any suitable number of pattern elements. Moreover,each pattern element may be configured to be changed between a firststate A1, B1, C1, D1, E1, F1, G1, H1 to at least a second state. In FIG.4 , pattern elements ENPa, ENPb change from a respective first state A1,B1 to a second state A2, B2, while pattern element ENPf and ENPh changefrom a first state F1, H1 to a corresponding third and tenth state F3,H10 respectively. The number of states may also be much more than thetwo, three or ten different states indicated in FIG. 4 . The number ofstates may be configured to be individually different amongst thepattern elements and may be configured to be automatically adjusteddepending on the ambient light conditions of the vehicle's surroundingor at the light emitting unit providing the encoded pattern. In oneembodiment, the different states are represented by differentcombinations of colors, for instance RGB colors.

FIG. 5 shows a flow chart of a method according to an embodiment of theinvention of communicating information between a vehicle 20 and a remoteterminal 10. The method S0 comprises the step S1 of providing at leastone image frame 11 representing at least a portion of the vehicle'ssurrounding. This step S1 of providing the at least one image frame iscarried out by means of one or more imaging sensors configured tocapture visual information about the vehicle's surrounding in the formof one or more image frame for every time instance. The at least oneimage frame 11 is provided so that an encoded pattern ENP1 is visible onthe at least one image frame 11, which encoded pattern ENP1 representspredetermined information to be transmitted to the remote terminal. Asmentioned previously, the encoded pattern may be provided by a lightemitting device arranged in the field of view of the one or more imagingsensors, which light emitting device is configured to display theencoded pattern. In some embodiments, the encoded pattern is changedover time. For instance, the imaging sensors are configured to capturefor every predetermined time instance at least one image frame 11. Themethod S0 further comprises the step S2 of transmitting the at least oneimage frame 11 from the vehicle 20 to the remote terminal 10. At theremote terminal, the at least one image frame 11 is displayed. Themethod S0 further comprises the step of reading S3, at the remoteterminal 10, the encoded pattern ENP2 visible on the at least one imageframe 11. The method S0 further comprises a step of decoding S4 thepredetermined information from the encoded pattern read at the terminal.The decoding is performed either at the remote terminal 10 or in thevehicle 20 when the read encoded pattern has been transmitted back tothe vehicle 20. In some embodiments, the predetermined information ofthe encoded pattern ENP1, ENP2, ENP3 includes a timestamp of the atleast one image frame, or the predetermined information of the encodedpattern ENP1, ENP2 may be solely a timestamp of the at least one imageframe indicating the time instance at which the at least one image framewas captured by the one or more imaging sensors. By communicatingtimestamps of each image frame in this manner, it also enables animage-to-image latency to be determined.

Moreover, more than just image frames may be communicated from thevehicle 20 to the remote terminal 10. As shown in FIG. 6 , the method S0comprises in some embodiments a step of transmitting S2′ sensory dataincluding the at least one image frame 11 from the vehicle 10 to theremote terminal 20. Based on this sensory data, a representation may beconstructed and displayed at the remote terminal 10. An operator at theremote terminal 10 may thus issue driving instructions based on thetransmitted sensory data, which is used to construct a representation atthe remote terminal 10 and based on the decoded predeterminedinformation. By means of a step S5, the driving instructions may betransmitted from the remote terminal 10 to the vehicle 20.

The vehicle 20 may then verify that driving instructions are accompaniedwith an encoded pattern representing transmitted predeterminedinformation or the predetermined information directly, whichpredetermined are also corresponding to previously transmittedpredetermined information. As shown in FIG. 6 , the method S0 maycomprise a step of determining S6 an image-to-image latency or around-trip time between the vehicle 20 and the remote terminal. This maybe based on the decoded predetermined information of the at least oneimage frame. The determined image-to-image latency or round-trip timemay be compared to a maximally tolerable image-to-image latency orround-trip time predetermined based on specific requirements and thecommunication established between the vehicle 20 and the remote terminal10. If the determined image-to-image latency or round-trip time fails tosatisfy a predetermined image-to-image latency condition or apredetermined round-trip time condition, an emergency fail-safeprocedure of the vehicle may be executed. This step of executing S7 anemergency fail-safe procedure of the vehicle is also shown in FIG. 6 .

FIGS. 7 and 8 shows a flow chart of a select collection of method stepsaccording to an embodiment of the invention, namely method steps S9, S9′and S10, S10′ of the method S0 shown in FIG. 6 . The encoded patternENP1, ENP2, ENP3 may be used to generate S9, S9′ a messageauthentication code MAC1, MAC2, MAC3 which are used S10, S10′ toauthenticate communication transmitted between the vehicle 20 and theremote terminal 10. The different notations S9, S9′, S10, S10′ denotethe direction of communication between the vehicle 20 and the remoteterminal; steps S9, S10 refer to communication going from vehicle 20 tothe remote terminal 10 and steps S9′, S10′ refer to communication goingfrom remote terminal 10 to vehicle 20.

In each direction, authentication of communication is enabled by meansof a shared message authentication code algorithm MAC-A and a sharedmessage authentication code key K. At the vehicle 20, the originalencoded pattern ENP1 in the at least one image frame 11 is used as aninput together with the shared message authentication code key K togenerate a first message authentication code MAC1. The first messageauthentication code MAC1 is then transmitted together with the sensorydata including the at least one image frame 11 and the original encodedpattern ENP1. The information is received at the remote terminal 10 andthe at least one image frame 11 is displayed and the transmitted encodedpattern ENP2 is read at the remote terminal. This transmitted encodedpattern ENP2 may be different due to an inadequate representation of thevehicle's surrounding. The transmitted encoded pattern ENP2 is usedtogether with the shared message authentication code key K in the sharedmessage authentication code algorithm MAC-A to generate a second messageauthentication code MAC2. Then the first and second messageauthentication codes MAC1, MAC2 are compared to see if they areidentical or not. If not identical, it may be deduced that the at leastone image frame may have been inadequately transmitted from the vehicle20 to the remote terminal 10.

In the opposite direction, when for example driving instructions are tobe sent to the vehicle, the second message authentication code MAC2 istransmitted together with the driving instructions including thetransmitted encoded pattern ENP2 to the vehicle 20. The received encodedpattern ENP3 is then used together with the shared messageauthentication code key K in the shared message authentication algorithmMAC-A to generate a third message authentication code MAC3 which iscompared with the second message authentication code MAC2 to see if theyare identical or not. If changed, the vehicle may be configured torefuse the driving instructions.

In the drawings and specification, there have been disclosed preferredembodiments and examples of the invention and, although specific termsare employed, they are used in a generic and descriptive sense only andnot for the purpose of limitation, the scope of the invention being setforth in the following claims.

1. A method for communicating information between a vehicle and a remoteterminal, the method comprising the steps of: providing at least oneimage frame representing at least a portion of the vehicle and/or thevehicle's surrounding, wherein an encoded pattern (ENP1) representingpredetermined information to be transmitted to the remote terminal,generated by a light emitting device, is visible on the at least oneimage frame; transmitting the at least one image frame from the vehicleto the remote terminal; reading, at the remote terminal, the encodedpattern (ENP2) visible on the at least one image frame; and decoding thepredetermined information from the encoded pattern (ENP2) read at theremote terminal.
 2. The method according to claim 1, wherein thepredetermined information of the encoded pattern (ENP1, ENP2, ENP3)includes a timestamp of the at least one image frame.
 3. The methodaccording to claim 1, further comprising the steps of transmittingsensory data including the at least one image frame from the vehicle tothe remote terminal; and transmitting driving instructions from theremote terminal to the vehicle together with the decoded predeterminedinformation and/or the read encoded pattern (ENP2).
 4. The methodaccording to claim 1, further comprising the step of determining animage-to-image latency or a round-trip time between the vehicle and theremote terminal based on the decoded predetermined information of the atleast one image frame.
 5. The method according to claim 4, furthercomprising the step of: executing an emergency fail-safe procedure ofthe vehicle when the image-to-image latency or the round-trip time failsto satisfy a predetermined image-to-image latency condition or apredetermined round-trip time condition.
 6. The method according toclaim 1, further comprising the steps of: generating a messageauthentication code (MAC1, MAC2, MAC3) based on the encoded pattern(ENP1, ENP2, ENP3) of at least one image frame, and using the messageauthentication code (MAC1, MAC2, MAC3) to authenticate communicationtransmitted between the vehicle and the remote terminal.
 7. The methodaccording to claim 1, wherein the encoded pattern (ENP) comprises a setof pattern elements (ENPa, ENPb) configured so that each pattern element(ENPa, ENPb) may be set to either of at least a first state and a secondstate.
 8. The method according to claim 7, wherein the encoded pattern(ENP) is a color pattern, and the at least a first state and a secondstate of each pattern element (ENPa, ENPb) are of different colors. 9.The method according to claim 1, wherein the encoded pattern (ENP) isgenerated by the light emitting device is placed in the field of view ofan imaging device of the vehicle configured to capture the sensory dataused for generating the at least one image frame.
 10. The methodaccording to claim 7, wherein the light emitting device comprises aplurality of light emitting diodes configured to pattern elements (ENPa,ENPb) of the encoded pattern.
 11. The method according to claim 7,wherein the hue, brightness and/or saturation of available colors ofeach pattern element (ENPa, ENPb) is adjusted based on ambient lightconditions of the vehicle.
 12. The method according to claim 1, whereinthe encoded pattern (ENP1) is read at the remote terminal by means of anencoded pattern detection device.
 13. The method according to claim 1,wherein the encoded pattern (ENP) is encrypted, and the encoded pattern(ENP) is decrypted using an encryption key.
 14. A communications systemcomprising a first system part for use in a vehicle and a second systempart for use in a remote terminal, the first and second system partconfigured to enable the method for communicating information betweenthe vehicle and the remote terminal according to claim
 1. 15. A vehiclesystem comprising one or more vehicles, at least one remote terminal,and a communications system according to claim
 14. 16. The methodaccording to claim 8, wherein the light emitting device comprises aplurality of light emitting diodes configured to pattern elements (ENPa,ENPb) of the encoded pattern.